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This is in response to the appeal brief filed May 21 , 2008 appealing from the Office 
action mailed March 4, 2008. 

(2) Related Appeals and Interferences 

The examiner is not aware of any related appeals, interferences, or judicial 
proceedings which will directly affect or be directly affected by or have a bearing on the 
Board's decision in the pending appeal. 

(3) Status of Claims 

The statement of the status of claims contained in the brief is correct. 

(4) Status of Amendments After Final 

The appellant's statement of the status of amendments after final rejection 
contained in the brief is correct. 

(5) Summary of Claimed Subject Matter 

The summary of claimed subject matter contained in the brief is correct. 

(6) Grounds of Rejection to be Reviewed on Appeal 

The appellant's statement of the grounds of rejection to be reviewed on appeal is 
correct. 

(7) Claims Appendix 

The copy of the appealed claims contained in the Appendix to the brief is correct. 

(8) Evidence Relied Upon 

6,311,150 Ramaswamy et al. 10-2001 

4,974,191 Amirgohdsi et al. 11-1990 
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Ramaswamy et al., " A pervasive Conventional Interface for information 
Interaction" Eurospeech 99, Budapest, Hungary ( 1999 ) pp 1-4 

(9) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

Claims 1,4,1 0-12, 1 5, 21-23 and 26 are rejected under 35 U.S.C. 1 02(b) as 
being anticipated by Ramaswamy (6,311,150). 

As per claim 1, Ramaswamy discloses a method for recognizing commands in natural 
language, comprising the steps of: 

Comparing an utterance to a plurality of handlers (column 2 lines 25-65, an input 
utterance is translated using a natural language understanding engine comprised of a 
plurality of translator levels. Each level contains a plurality of categories with associated 
formal language commands stored as models (handlers). The categories with 
associated formal language commands are compared to the input utterance); 
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Identifying a winning handler for decoding a command from the utterance, 
wherein the winning handler is identified by arbitration between results provided by at 
least two of the plurality of handlers, and the results provided at a first stage by at least 
one of the at least two of the plurality of handlers include one or more target utterances 
for the utterance (column 2 lines 25-65, scores for the probability of a correct translation 
are determined for the categories and associated formal commands at each translator 
level (handler). The translator levels that do not output a do nothing command (winning 
handler) are used to determine the final formal command and output it at the last 
translator level (first stage)); and 

Decoding the command in accordance with the winning handler (column 2 lines 
25-65, scores for the probability of a correct translation are determined for the 
categories, then the highest scoring category, including the associated formal 
command, are passed to the next translator level. The translator levels that do not 
output a do nothing command (winning handler) are used to determine the final formal 
command and output it at the last translator level (first stage)). 

As per claim 12, Ramaswamy discloses a method for recognizing commands in natural 
language, comprising the steps of: 

Providing a plurality of handlers trained to be responsive to given utterances 
(column 2 lines 25-65 and column 3 lines 10-20, an input utterance is translated using a 
natural language understanding engine comprised of a plurality of translator levels. 
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Each level contains a plurality of categories with associated formal language commands 
stored as models (handlers). The categories with the associated formal language 
commands are compared to the input utterance, where the categories are trained from 
training data from domain inputs); 

Arbitrating against results provided by at least two of the plurality of handlers to 
determine a winning handler for an utterance, wherein the results provided at a first 
stage by at least one of the at least two of the plurality of handlers include one or more 
target utterances for the utterance (column 2 lines 25-65, scores for the probability of a 
correct translation are determined for the categories and associated formal commands 
at each translator level (handler). The translator levels that do not output a do nothing 
command (winning handler) are used to determine the final formal command and output 
it at the last translator level (first stage)); and 

Decoding the command in accordance with the winning handler (column 2 lines 
25-65, scores for the probability of a correct translation are determined for the 
categories and associated formal commands at each translator level (handler). The 
translator levels that do not output a do nothing command (winning handler) are used to 
determine the final formal command and output it at the last translator level (first 
stage)). 

As per claim 23, Ramaswamy discloses a system for recognizing commands in natural 
language, comprising: 
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A speech recognizer for decoding language and semantic information in 
utterances provided by a user (column 4 lines 61-67); and 

A dialog manager comprising a hierarchical ordering of handlers, each handler 
being trained to be responsive to decoded utterances wherein the dialog manager 
manages arbitration between results provide by the handlers to determine a winning 
handler for an utterance and decodes the command in accordance with the winning 
handler, wherein the results provided at a first stage include one or more target 
utterances for the utterance (column 2 lines 25-65 and column 3 lines 10-20, an input 
utterance is translated using a natural language understanding engine comprised of a 
plurality of translator levels. Each level contains a plurality of categories with associated 
formal language commands stored as models (handlers). The categories with the 
associated formal language commands are compared to the input utterance, where the 
categories are trained from training data from domain inputs. Scores for the probability 
of a correct translation are determined for the categories and associated formal 
commands at each translator level (handler). The translator levels that do not output a 
do nothing command (winning handler) are used to determine the final formal command 
and output it at the last translator level (first stage)) 

As per claims 4, 15 and 26, Ramaswamy discloses the method as recited in claims 1, 
12 and 23, wherein the handlers include an enabled or a disabled state and further 
comprising the step of presenting the utterance to enabled handlers (column 8 lines 25- 
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44), the utterance is compared to categories at translator levels, therefore it is inherent 
that they are enabled). 

As per claims 10 and 21 , Ramaswamy discloses the method as recited in claims 1 and 
12, wherein the step of decoding further includes executing a command in accordance 
with the winning handler, responsive to the utterance (column 2 lines 25-40, scores for 
the probability of a correct translation are determined for the categories, then the 
category and corresponding formal language command having the highest score is 
chosen, and passed to the next translator level. After the last level of translation, the 
formal language command is output to control an application). 

As per claims 1 1 and 22, Ramaswamy discloses a computer-readable medium, 
tangibly embodying a program of instructions executable by a computer to perform 
method step for recognizing commands in natural language as recited in claims 1 and 
12 (column 1 lines 40-46). 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 5 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ramaswamy. 

Ramaswamy discloses the method as recited in claims 4 and 15, but Ramaswamy 
does not disclose submitting the utterance to disabled container handlers to ensure 
submission of the utterance to child handlers. However, Ramaswamy does disclose 
hierarchical translators, where each level performs one portion of the translation and 
narrows the search space of formal commands for the subsequent levels (column 5 
lines 40-53). Therefore each current level determines enabled or disabled categories, 
and thus formal commands, for use in the next level. Ramaswamy is also concerned 
with developing a robust system using techniques to improve the accuracy of the NLU 
system by checking for translation errors from previous levels (column 8 lines 25 - 
column 9 lines 40). Submitting the utterance to disabled handlers is one method of 
checking for translation errors, since an utterance correctly translated by child handlers 
of a disabled state and incorrectly translated by child handlers of an enabled state 
indicates a possible translation error in a parent handler. 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to submit the utterance to disabled container handlers to ensure 
submission to child handlers in Ramaswamy, since it would enable the system to check 
for translation errors, thus improving system performance. 
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Claims 2, 3, 6, 13, 14, 17, 24 and 25 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Ramaswamy in view of Amirghodsi (4,974,191 ). 

As per claims 2, 13 and 24, Ramaswamy discloses the method as recited in claims 1, 
12 and 23, however Ramaswamy does not disclose wherein the step of identifying 
includes resolving ties in the arbitration between handlers by employing a tie-breaker 
handler. Amirghodsi discloses a system that classifies objects of speech into classes 
and resolves a deadlock or tie when it occurs (column 39 lines 11-15). Amirghodsi 
discloses a natural language translation system for a human/computer interface 
(column 2 lines 15-18), and is therefore analogous art. 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to resolve ties in the arbitration between handlers by employing a tie- 
breaker handler in Ramaswamy, since it would enable the system to continue 
processing after reaching a deadlock, as indicated in Amirghodsi (column 39 lines 11- 
15), thus providing quick resolution of a tie, and increased processing speed. 

As per claims 3, 14 and 25, Ramaswamy in view of Amirghodsi disclose the method 
as recited in claims 2, 13 and 24, however Ramaswamy does not disclose wherein the 
tie-breaker handler poses a question to a user to determine the winning handler. 
Amirghodsi discloses a system that classifies objects of speech into classes and 
resolves a deadlock or tie when it occurs (column 39 lines 11-15), as well as a system 
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that uses questions addressed to the user to gain further information in order to process 
a user request (column 7 lines 44-55 and 59-62). Amirghodsi discloses a natural 
language translation system for a human/computer interface (column 2 lines 15-18), 
and is therefore analogous art. 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to have a tie-breaker handler pose a question to a user to determine the 
winning handler in Ramaswamy, since it would enable the system to gather further 
information, as indicated in Amirghodsi (column 7 lines 59-62), which would then be 
used to determine the correct handler, thus providing quick resolution of a tie, and 
increased processing speed. 

As per claims 6 and 17, Ramaswamy discloses the method as recited in claims 1 and 
12, however Ramaswamy does not disclose further comprising the step of submitting 
unresolved utterances to winning handlers of a previous utterance for decoding. 
Amirghodsi discloses a system that classifies objects of speech into classes and 
resolves a deadlock or tie when it occurs by assigning the last class reference to the 
remaining objects (column 39 lines 11-15). Amirghodsi discloses a natural language 
translation system for a human/computer interface (column 2 lines 15-18), and is 
therefore analogous art. 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to submit unresolved utterances to winning handlers of a previous 
utterance for decoding in Ramaswamy, since it would enable the system to continue 
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processing in the case of a deadlock or tie, as indicated in Amirghodsi (column 39 
lines 11-15). 

Claims 7-9, 18-20 and 27-29 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Ramaswamy in view of Ramaswamy2 ("A Pervasive Conservational 
interface for information interaction" Eurospeech 99). 

As per claims 7, 18 and 27, Ramaswamy discloses the method as recited in claims 1, 
12 and 23, however Ramaswamy does not disclose the step of maintaining a database 
of a history of handler selections. Ramaswamy2 discloses a system that maintains a 
database of a history of handler selections (section 2.2 Conversational System, third 
paragraph, a multimodal history captures all conversational and graphical system 
events, and keeps track of the system state). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to maintain a database of a history of handler selections in 
Ramaswamy, in order to improve the natural language understanding system for the 
predictable result of enabling standard disambiguation and reference resolution, 
especially when many transactions are open, as indicated in Ramaswamy2 (section 2.2 
Conversational System, third and fifth paragraphs). 
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As per claims 8, 19 and 28, Ramaswamy in view of Ramaswamy2 disclose the method 
as recited in claims 7, 18 and 27, however Ramaswamy does not disclose wherein the 
history includes time based ordering and ontological information. Ramaswamy2 
discloses wherein the history includes time based ordering and ontological information 
(section 2.2 Conversational System, third paragraph, a multimodal history captures all 
conversational and graphical system events, and keeps track of the system state). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use a history that includes time based ordering and ontological 
information in Ramaswamy, in order to improve the natural language understanding 
system for the predictable result of enabling standard disambiguation and reference 
resolution, especially when many transactions are open, as indicated in Ramaswamy2 
(section 2.2 Conversational System, third and fifth paragraphs). 

As per claims 9,20 and 29, Ramaswamy in view of Ramaswamy2 disclose the method 
as recited in claims 7,18 and 27, however Ramaswamy does not disclose the step of 
resolving unresolved utterances by employing information stored in the database. 
Ramaswamy2 discloses resolving unresolved utterances by employing information 
stored in the database (section 2.2 Conversational System, sixth paragraph, the 
multimodal history is used for disambiguation and reference resolution). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use the information stored in the database to resolve unresolved 
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utterances in Ramaswamy, in order to improve the natural language understanding 
system for the predictable result of enabling standard disambiguation and reference 
resolution especially when many transactions are open, as indicated in Ramaswamy2 
(section 2.2 Conversational System, third and fifth paragraphs). 

(10) Response to Argument 

Appellant argues that Ramaswamy does not disclose the elements of 
independent claims 1,12 and 23, more specifically stating that, "Ramaswamy doe not 
teach or suggest the preceding limitations of claims 1,12 and 23, but instead involves a 
multi-tiered approach that, at a first stage, only outputs a target category for an 
utterance, but not a target utterance" (Appeal Brief page 13); however the examiner 
respectfully disagrees. It appears that appellant is equating each translator level in 
Ramaswamy (see Figure 2, item 102, translator levels 1- N) with a stage, and thus the 
first translator level with a first stage. However, the examiner notes that this is not the 
only reasonable interpretation available. In Ramaswamy the input command is passed 
to translator levels (handlers), where each level interprets the utterance and assigns it 
to a category and corresponding formal language command(s). Once the input has 
been passed through the translator levels (handlers) a formal command is output at the 
final translator level, or first stage (Figure 2, and column 2 lines 25-65). As noted by 
appellant, Ramaswamy discloses that each translator level performs a portion of the 
translation process, with the N translator concluding the process and outputting the 
formal command (column 5 lines 43-50). Ramaswamy does disclose a hierarchical 
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translation process; however, the hierarchical translation steps combine to output a 
formal command at a first stage at the final translator level. Thus Ramaswamy 
discloses "the results provided at a first stage by at least one of the at least two of the 
plurality of handlers include one or more target utterance for the utterance" as recited in 
claims 1, 12 and 23. 

Appellant also argues that, "Ramaswamy does not teach or suggest the use of 
arbitration at all, but rather simply selects the category with the highest score." (Appeal 
brief page 16). Appellant also asserts that, "in arbitration, a person or thing acts as a 
judge to make a choice based on discretion"(Appeal brief page 17), this usage of 
arbitration based on the definition provided by Webster's New World Dictionary of the 
American Language. However, the examiner continues to assert that this interpretation 
of arbitration is not supported in the specification. Appellant has pointed to page 14 of 
the specification as allegedly providing support for the argument that "arbitration" is 
"discretionary", however the examiner notes that this is only part of disclose describing 
arbitration. The specification, page 1 1 , states that, "The handlers 20 decide among 
themselves what the intended target is by comparing features in the utterance to 
content stored in each handler to determine a highest score (which may include 
weighting and other score modification techniques, which may be known in the art). The 
handlers 20 are organized into a hierarchy as illustratively shown in FIG. 2. A database 
204 is included to provide additional information in determining a winning handler for 
execution of commands". The specification also states that, "Each handler, in its 
definition, is provided with a list of utterances it may understand. Since the utterance as 
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represented here, is a semantic parse, such a list of utterances is actually a list of 
concepts and may be defined quite concisely" (Specification page 13), we well as that 
questions are posed to each handler, as stated by appellant; however the specification 
continues to state that, "each enabled handler responds in the affirmative or the 
negative" to the questions posed to it (page 13). Thus, winning handlers, to be used to 
execute commands, are determined with the use of binary responses to questions, the 
responses determined by scoring amongst the handlers; these methods do not support 
the interpretation of arbitration, as argued by appellant. 

Appellant also argues that, "while claims 1,12 and 23 recite 
identifying/determining a winning handler , the cited portion of Ramaswamy discloses 
selecting a category (and not a winning handler as recited in these claims)", however 
the examiner respectfully disagrees. In Ramaswamy, the input command is passed to 
translator levels, each level then interpreting the utterance and assigning it to a category 
and corresponding formal language command(s) (winning handler), based on a 
probability of correct translation for each of the categories associated with that translator 
level. If a translator error is encountered, a do nothing category is selected (column 2 
lines 59-65). Upon output at the last translator level, the translator level (handler) with a 
do nothing output does not contribute the determination of a final formal command. 
Thus Ramaswamy discloses, "identifying/determining a winning handler" as stated in 
claims 1,12 and 23, where the winning handlers are translator levels that output a 
category and corresponding formal command, as opposed to a do nothing command. 
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Appellant also argues that Ramaswamy does not disclose or suggest that the, 
"command is decoded in accordance with the winning handler identified by arbitration" 
since, "Ramaswamy does not output a formal language command corresponding to the 
handler (translator level) but rather then associates the selected category with a next 
level of translators and ultimately output the formal language command from a last level 
of translators." (Remarks page 14). However, the examiner respectfully disagrees. In 
Ramaswamy, the input command is passed to translator levels, each level then 
interpreting the utterance and assigning it to a category and corresponding formal 
language command(s) (winning handler), the category determined based on the 
probability of a correct translation. The identified category and corresponding formal 
command (winning handler) is then used in the next level of continued translation. If a 
translator error is encountered, a do nothing category is selected (column 2 lines 59- 
65). Upon output at the last translator level, the translator level (handler) with a do 
nothing output does not contribute the determination of a final formal command. 
Therefore Ramaswamy discloses "decoding the command in accordance with the 
winning handler" as recited in independent claim 1,12 and 23. 

(11) Related Proceeding(s) Appendix 

No decision rendered by a court or the Board is identified by the examiner in the 
Related Appeals and Interferences section of this examiner's answer. 



For the above reasons, it is believed that the rejections should be sustained. 
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Respectfully submitted, 
Dorothy Siedler 

Conferees: 

Richemond Dorvil/R. D.I 

Supervisory Patent Examiner, Art Unit 2626 

Patrick Edouard 
/Patrick N. Edouard/ 

Supervisory Patent Examiner, Art Unit 2626 
/Richemond Dorvil/ 

Supervisory Patent Examiner, Art Unit 2626 



